Skip to main content

ArchiveBot: The Archive Team Crowdsourced Crawler

ArchiveBot is an IRC bot designed to automate the archival of smaller websites (e.g. up to a few hundred thousand URLs). You give it a URL to start at, and it grabs all content under that URL, records it in a WARC, and then uploads that WARC to ArchiveTeam servers for eventual injection into the Internet Archive (or other archive sites).

To use ArchiveBot, drop by #archivebot on EFNet. To interact with ArchiveBot, you issue commands by typing it into the channel. Note you will need channel operator permissions in order to issue archiving jobs. The dashboard shows the sites being downloaded currently.

There is a dashboard running for the archivebot process at http://www.archivebot.com.

ArchiveBot's source code can be found at https://github.com/ArchiveTeam/ArchiveBot.

16,724
RESULTS
rss


PART OF
Archive Team
Media Type
16,724
web
Year
283
2019
1,755
2018
9,394
2017
3,909
2016
942
2015
424
2014
More right-solid
Topics & Subjects
12,584
archivebot
1
184.180.244.41
1
3dblogger.typepad.com
1
ahkscript.org
1
approachingaro.org
1
arstechnica.com
More right-solid
Collection
More right-solid
Creator
4,132
archive team
Language
4
English
SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 125.4M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 31.7M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 7.6M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
Topic: archivebot
Topic: archivebot
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 2.1M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.8M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.8M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.3M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1.3M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 1M
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 990,271
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 953,377
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 891,258
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 856,149
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 813,994
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 789,947
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 756,589
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 738,666
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 703,667
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 673,804
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 650,701
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 616,665
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 612,553
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 604,210
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 579,575
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 577,497
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 573,660
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 570,401
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 568,775
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 551,111
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 542,840
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 533,166
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 532,341
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 527,006
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 522,906
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 514,243
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 508,302
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 507,713
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 505,525
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 504,189
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 503,484
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 489,090
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 486,023
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 483,395
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 482,064
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 470,331
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 470,160
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 464,029
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 458,721
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 451,551
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 451,396
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 448,530
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 446,400
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 443,137
favorite 1
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 442,412
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 441,989
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
Topic: archivebot
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 435,163
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 435,065
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 433,436
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 432,234
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.
ArchiveBot: The Archive Team Crowdsourced Crawler
by Archive Team
web
eye 422,964
favorite 0
comment 0
ArchiveBot is an Archive Team service to quickly grab smaller at-risk or critical sites to bring copies into the Internet Archive Wayback machine.